Model Switching by Channel Fusion for Network Pruning and Efficient Feature Extraction

The paper introduces a feature dimension reduction method called channel fusion, and a criterion for redundant channel detection named effective map distance. Channel fusion locally reduces the feature dimension by replacing the redundant channel pair with a single channel, suppressing the map distance between the two models. It is applicable to network model switching such as pruning hidden layer units and reducing input channels. Effective map distance is a measure of discrepancy in the models before and after the channel reduction, which can be defined for any dimension reduction strategy. The two methods were applied to the feature extraction layer of a network for image texture classification. Improvements both in the classification rate and the training speed were observed when the methods were used during the training, which dynamically enabled to switch the model for efficient feature extraction.